Dataset overview

At the moment LOBSTER data available for 1520 compounds.

All the computational data (including raw unprocessed data) is available to download from our Zenodo communuity : https://zenodo.org/communities/automated-bonding-analysis/?page=1&size=20

Dataset size:

• Unprocess raw data : ~345 GB
• Lightweight jsons (Could be directly used for material project website) : ~ 716 MB
• Computational data in pydantic schema (as per atomate2 workflow) : ~80 GB

Top level keys to access data from LOBSTER lightweight JSON files

Root Keys	Data type	Description
all_bonds	dict	Summarized relevant bonds data (See table 2 of the manuscript for details)
cation_anion_bonds	dict	Summarized relevant cation-anion bonds data (See table 2 of the manuscript forfor details)
madelung_energies	dict	Total electrostatic energy for the structure as calculated from the Mulliken and Löwdin charges
charges	dict	Atomic charges with Mulliken and Löwdin population analysis methods as keys. Each keys corresponding list follows the order of sites in the crystal structure.
calc_quality_summary	dict	Dict summarizing results of technical validation tests like charge spillings, band overlaps, DOS and charge comparisons

Top level keys to access data from Computational data JSON files

Root Keys	Data type	Description
structure	dict	Dict representation of the pymatgen structure object used for the LOBSTER calculation
charges	dict	Atomic charges dict from LOBSTER based on Mulliken and Loewdin charge analysis
lobsterin	dict	LOBSTER calculation inputs
lobsterout	dict	Information of LOBSTER calculation output
lobsterpy_data	dict	Summarized bonding analysis data from Lobsterpy (all bonds mode). It also includes Cohp objects to plot the COHP curves from the automatic analysis
lobsterpy_text	dict	LobsterPy automatic analysis summary text (all bonds mode)
strongest_bonds_icohp	dict	Describes the strongest ICOHP bonds
strongest_bonds_icoop	dict	Describes the strongest ICOOP bonds
strongest_bonds_icobi	dict	Describes the strongest ICOBI bonds
lobsterpy_data_cation_anion	dict	Summarized bonding analysis data from Lobsterpy (cation-anion bonds mode). It also includes Cohp objects to plot the COHP curves from the automatic analysis
lobsterpy_text_cation_anion	dict	LobsterPy automatic analysis summary text (cation-anion bonds mode)
strongest_bonds_icohp_cation_anion	dict	Describes the strongest cation-anion ICOHP bonds
strongest_bonds_icoop_cation_anion	dict	Describes the strongest cation-anion ICOOP bonds
strongest_bonds_icobi_cation_anion	dict	Describes the strongest cation-anion ICOBI bonds
cohp_data	dict	Dict representation of pymatgen CompleteCohp object including data to plot COHP curves
coop_data	dict	Dict representation of pymatgen CompleteCohp object including data to plot COOP curves
cobi_data	dict	Dict representation of pymatgen CompleteCohp object including data to plot COBI curves
dos	dict	Dict representation of pymatgen LobsterCompleteDos object including the DOSCAR.lobster data
lso_dos	dict	Dict representation of pymatgen LobsterCompleteDos object including the DOSCAR.LSO.lobster data
madelung_energies	dict	Consists of the Madelung energies of the structure derived from the Mulliken and Löwdin charges
site_potentials	dict	Site potentials dict based on Mulliken and Loewdin charges
gross_populations	dict	Gross populations dict based on Mulliken and Loewdin charges with each site as a key and the gross population as a value.
band_overlaps	dict	Band overlaps data for each k-point from bandOverlaps.lobster file if it exists

Parsers / Plotters - Pymatgen

Parsers for all the LOBSTER output files are already implemented in pymatgen namely :

• COHPCAR.lobster / COBICAR.lobster / COBICAR.lobster
• ICOHPLIST.lobster / ICOOPLIST.lobster / ICOBILIST.lobster
• DOSCAR.lobster / DOSCAR.LSO.lobster
• MadelungEnergies.lobster
• CHARGE.lobster

Plotters for DOSCAR, COHP(ICOHP)/COBI (ICOBI)/COOP (ICOOP) are also readily available. Pymatgen DOS plotter now also has option to plot DOS in convention followed in chemistry: https://github.com/materialsproject/pymatgen/blob/d621490d5d6742b4c7e432d3144dec6858236c79/pymatgen/electronic_structure/plotter.py#L131

Parsers / Plotters - Lobsterpy

Interactive plotter for COHP/ICOHP are being implemented in Lobsterpy package (Useful for website)

Scripts for generating and reading the output files - Lightweight JSONS / Computational data JSONS

• Data generation : https://github.com/naik-aakash/lobster-database-paper-analysis-scripts/tree/main/Data_generation
• Data reading : https://github.com/naik-aakash/lobster-database-paper-analysis-scripts/tree/main/Read_data_records

DOS comparison / Visualizer app

We have implemented methods in pymatgen that can easily compare DOS using Tanimoto index as similarity measure. (Could be useful to demonstrate it on website) :

• Fingerprint: https://github.com/materialsproject/pymatgen/blob/d621490d5d6742b4c7e432d3144dec6858236c79/pymatgen/electronic_structure/dos.py#L1180

• Compare two DOS fingerprints (Tanimoto index): https://github.com/materialsproject/pymatgen/blob/d621490d5d6742b4c7e432d3144dec6858236c79/pymatgen/electronic_structure/dos.py#L1284

• We also have created a dash app that helps in visualizing the DOS benchmark for our dataset and is available on zenodo : https://doi.org/10.5281/zenodo.7795903

Lightweight JSONS

import gzip
import json
from pymatgen.electronic_structure.cohp import Cohp
from pymatgen.electronic_structure.plotter import CohpPlotter
from lobsterpy.plotting import InteractiveCohpPlotter

Top level keys to access data from LOBSTER lightweight JSON files

Root Keys	Data type	Description
all_bonds	dict	Summarized relevant bonds data (See table 2 of the manuscript for details)
cation_anion_bonds	dict	Summarized relevant cation-anion bonds data (See table 2 of the manuscript forfor details)
madelung_energies	dict	Total electrostatic energy for the structure as calculated from the Mulliken and Löwdin charges
charges	dict	Atomic charges with Mulliken and Löwdin population analysis methods as keys. Each keys corresponding list follows the order of sites in the crystal structure.
calc_quality_summary	dict	Dict summarizing results of technical validation tests like charge spillings, band overlaps, DOS and charge comparisons.

def read_lobster_lightweight_json(filename: str):
    """
    This function loads the json.gz files and returns a dictionary with lobsterpy summmarized bonding analysis.
    
    Args:
        filename (str) : name of file
    Returns:
        Returns a dictionary with lobstepy summmarized bonding analysis output.
    """
    with gzip.open(filename, 'rb') as f:
        data = json.loads(f.read().decode('utf-8'))
        
    lobster_data = {}
    for item in data:
        lobster_data.update(item)
   
    return lobster_data

lobster_summary = read_lobster_lightweight_json(filename='Example_data/Lightweight_jsons/mp-1000.json.gz')

## Keys to access data
lobster_summary.keys()

dict_keys(['cation_anion_bonds', 'all_bonds', 'madelung_energies', 'charges', 'calc_quality_summary'])

Lobsterpy automatic analysis

Cation-anion mode

# Summarized bonding dictionary for cation-anion bonds 
lobster_summary['cation_anion_bonds']

# Recreate automatic analysis plots (cation-anion mode)
for number, (key, cohp) in enumerate(lobster_summary['cation_anion_bonds']["lobsterpy_data"]["cohp_plot_data"].items()):
    plotter = CohpPlotter()
    cohp = Cohp.from_dict(cohp)
    plotter.add_cohp(key, cohp)
    plotter.show(ylim=[-10,5], xlim=[-3,5])

# Recreate automatic analysis interactive plots (cation-anion mode)
iplotter = InteractiveCohpPlotter()
iplotter.add_cohps_from_plot_data(lobster_summary['cation_anion_bonds']["lobsterpy_data"]["cohp_plot_data"])
iplotter.get_plot(ylim=[-10,5], xlim=[-3,5])

All bonds mode

# Summarized bonding dictionary for all bonds 
lobster_summary['all_bonds']

# Recreate automatic analysis plots (all bonds)
for number, (key, cohp) in enumerate(lobster_summary['all_bonds']["lobsterpy_data"]["cohp_plot_data"].items()):
    plotter = CohpPlotter()
    cohp = Cohp.from_dict(cohp)
    plotter.add_cohp(key, cohp)
    plotter.show(ylim=[-10,5], xlim=[-3,5])

# Recreate automatic analysis interactive plots (all bonds)
iplotter = InteractiveCohpPlotter()
iplotter.add_cohps_from_plot_data(lobster_summary['all_bonds']["lobsterpy_data"]["cohp_plot_data"])
iplotter.get_plot(ylim=[-10,5], xlim=[-3,5])

#Atomic charges for the sites in input crystal structure 
lobster_summary['charges']

{'Mulliken': [1.58, -1.58], 'Loewdin': [1.49, -1.49]}

# Madelung energy of the structure
lobster_summary['madelung_energies']

{'Mulliken': -17.68, 'Loewdin': -15.69, 'Ewald_splitting': 3.14}

# Summary of calculation quality from technical validation tests 
lobster_summary['calc_quality_summary']

{'minimal_basis': True,
 'charge_spilling': {'abs_charge_spilling': 2.26},
 'band_overlaps': {'file_exists': False,
  'limit_maxDeviation': None,
  'has_good_quality_maxDeviation': True,
  'max_deviation': None,
  'percent_kpoints_abv_limit': None},
 'charges': {'BVA_Mulliken_agree': True, 'BVA_Loewdin_agree': True},
 'dos_comparisons': {'tanimoto_orb_s': 0.9715,
  'tanimoto_orb_p': 0.888,
  'tanimoto_summed': 0.9039,
  'e_range': [-15, 0],
  'n_bins': 256}}

Computational data JSONS

import os
import gzip
import ijson
from pymatgen.io.lobster import Lobsterout
from pymatgen.core import Structure
from pymatgen.electronic_structure.plotter import DosPlotter
from pymatgen.electronic_structure.cohp import Cohp
from pymatgen.electronic_structure.cohp import CompleteCohp
from pymatgen.electronic_structure.plotter import CohpPlotter
from pymatgen.electronic_structure.dos import LobsterCompleteDos
from atomate2.lobster.schemas import LobsterTaskDocument

Top level keys to access data from Computational data JSON files

Root Keys	Data type	Description
structure	dict	Dict representation of the pymatgen structure object used for the LOBSTER calculation
charges	dict	Atomic charges dict from LOBSTER based on Mulliken and Loewdin charge analysis
lobsterin	dict	LOBSTER calculation inputs
lobsterout	dict	Information of LOBSTER calculation output
lobsterpy_data	dict	Summarized bonding analysis data from Lobsterpy (all bonds mode). It also includes Cohp objects to plot the COHP curves from the automatic analysis
lobsterpy_text	dict	LobsterPy automatic analysis summary text (all bonds mode)
strongest_bonds_icohp	dict	Describes the strongest ICOHP bonds
strongest_bonds_icoop	dict	Describes the strongest ICOOP bonds
strongest_bonds_icobi	dict	Describes the strongest ICOBI bonds
lobsterpy_data_cation_anion	dict	Summarized bonding analysis data from Lobsterpy (cation-anion bonds mode). It also includes Cohp objects to plot the COHP curves from the automatic analysis
lobsterpy_text_cation_anion	dict	LobsterPy automatic analysis summary text (cation-anion bonds mode)
strongest_bonds_icohp_cation_anion	dict	Describes the strongest cation-anion ICOHP bonds
strongest_bonds_icoop_cation_anion	dict	Describes the strongest cation-anion ICOOP bonds
strongest_bonds_icobi_cation_anion	dict	Describes the strongest cation-anion ICOBI bonds
cohp_data	dict	Dict representation of pymatgen CompleteCohp object including data to plot COHP curves
coop_data	dict	Dict representation of pymatgen CompleteCohp object including data to plot COOP curves
cobi_data	dict	Dict representation of pymatgen CompleteCohp object including data to plot COBI curves
dos	dict	Dict representation of pymatgen LobsterCompleteDos object including the DOSCAR.lobster data
lso_dos	dict	Dict representation of pymatgen LobsterCompleteDos object including the DOSCAR.LSO.lobster data
madelung_energies	dict	Consists of the Madelung energies of the structure derived from the Mulliken and Löwdin charges
site_potentials	dict	Site potentials dict based on Mulliken and Loewdin charges
gross_populations	dict	Gross populations dict based on Mulliken and Loewdin charges with each site as a key and the gross population as a value.
band_overlaps	dict	Band overlaps data for each k-point from bandOverlaps.lobster file if it exists

Function below can reads one Root key data from the json at a time. Due to file large sizes it is recommended approach to avoid running of out memory issues

def read_lobstertask_doc_json(filename: str,pymatgen_objs: bool=False, query='structure'):
    """
    This function loads the lobster task json.gz files and returns a dictionary corresponding to query.
    
    Args:
        filename (str) : name of file
        pymatgen_objs (bool) : if True will convert structure,coop,cobi,cohp and dos to pymatgen objects
        query (str) : Filters the json file based on this key and returns data corresponding to it.  
        Possible query corresponds to root keys column of the table above.
    Returns:
        Returns a dictionary with lobster task json data corresponding to query.
    """
    lobster_data={}
    with gzip.open(filename, 'rb') as f:
        objects = ijson.items(f,'item',use_float=True)
        #filtered_objects = (obj for obj in objects if list(obj.keys())[0]==query)
        for obj in objects:
            if query in obj:
                for k, v in obj.items():
                    if pymatgen_objs:
                        if k=='structure':
                            lobster_data[k]=Structure.from_dict(v)
                        elif k=='lobsterpy_data' or k=='lobsterpy_data_cation_anion':
                            lobster_data[k]=v
                            if lobster_data[k]:
                                for plotlabel, cohp in lobster_data[k]["cohp_plot_data"].items():
                                    lobster_data[k]["cohp_plot_data"][plotlabel]= Cohp.from_dict(cohp)
                        elif k=='cohp_data' or k=='cobi_data' or k=='coop_data':
                            lobster_data[k]=CompleteCohp.from_dict(v)
                        elif k=='lso_dos' or k=='dos':
                            lobster_data[k]=LobsterCompleteDos.from_dict(v)
                        else:
                            lobster_data[k]=v
                    else:
                        lobster_data[k]=v
                break
                    
    return lobster_data

Section below provides some example snippets of how to access some data from the computational data json based on keys

#get structure
lobtask= read_lobstertask_doc_json(filename='Example_data/Computational_data_jsons/mp-1008223.json.gz', 
                                   pymatgen_objs=True, query='structure')
lobtask['structure']

Structure Summary
Lattice
    abc : 4.666904755831213 4.666904755831213 4.666904755831213
 angles : 59.99999999999999 59.99999999999999 59.99999999999999
 volume : 71.874
      A : 0.0 3.3 3.3
      B : 3.3 0.0 3.3
      C : 3.3 3.3 0.0
    pbc : True True True
PeriodicSite: Ca (0.0000, 0.0000, 0.0000) [0.0000, 0.0000, 0.0000]
PeriodicSite: Se (1.6500, 1.6500, 1.6500) [0.2500, 0.2500, 0.2500]

#get lobsterpy summary analysis summary text
lobtask= read_lobstertask_doc_json(filename='Example_data/Computational_data_jsons/mp-1008223.json.gz', 
                                   pymatgen_objs=True, query='lobsterpy_text')
lobtask['lobsterpy_text']

'The compound CaSe has 2 symmetry-independent atoms(s) with relevant bonds: Ca1, Se2. Ca1 has a tetrahedral (CN=4) coordination environment. It has 4 Ca-Se (mean ICOHP: -0.83 eV, 10.922 percent antibonding interaction below EFermi) bonds. Se2 has a tetrahedral (CN=4) coordination environment. It has 4 Se-Ca (mean ICOHP: -0.83 eV, 10.922 percent antibonding interaction below EFermi) bonds.'

#get lobster cohpcar pymatgen object and plot cohp by label
lobtask= read_lobstertask_doc_json(filename='Example_data/Computational_data_jsons/mp-1008223.json.gz', 
                                   pymatgen_objs=True, query='cohp_data')
cohppplot = CohpPlotter()
cohppplot.add_cohp(label='17',cohp=lobtask['cohp_data'].get_cohp_by_label('17')) #plot cohp by bond label
plot = cohppplot.get_plot(integrated=False)
plot.ylim([-12, 2])
plot.xlim([-1, 2])
plot.show()

#get lobster dos pymatgen object and plot projected dos
lobtask= read_lobstertask_doc_json(filename='Example_data/Computational_data_jsons/mp-1008223.json.gz', 
                                   pymatgen_objs=True, query='lso_dos')
plotter = DosPlotter()
plotter.add_dos_dict(lobtask['lso_dos'].get_spd_dos())
plotter.get_plot(xlim=(-15, 0));

#get lobster gross populations
lobtask= read_lobstertask_doc_json(filename='Example_data/Computational_data_jsons/mp-1008223.json.gz', 
                                   pymatgen_objs=True, query='gross_populations')
print(lobtask['gross_populations'])

{'0': {'element': 'Ca', 'Mulliken GP': {'3s': 2.0, '4s': 0.49, '3p_y': 2.0, '3p_z': 2.0, '3p_x': 2.0, 'total': 8.49}, 'Loewdin GP': {'3s': 1.99, '4s': 0.58, '3p_y': 2.0, '3p_z': 2.0, '3p_x': 2.0, 'total': 8.57}}, '1': {'element': 'Se', 'Mulliken GP': {'4s': 1.92, '4p_y': 1.86, '4p_z': 1.86, '4p_x': 1.86, 'total': 7.51}, 'Loewdin GP': {'4s': 1.85, '4p_y': 1.86, '4p_z': 1.86, '4p_x': 1.86, 'total': 7.43}}}